Directed Information and Conditional Mutual Information
نویسنده
چکیده
We study directed information in Bayesian networks and related structures. Mutual information is split into directed information and residual information. Some basic equations for directed information and residual information are determined.
منابع مشابه
Using Conditional Mutual Information to Approximate Causality for Multivariate Physiological Time Series
Causality analytic techniques based on conditional mutual information are described. Causality analysis may be used to infer linear and nonlinear causal relations between selected brain regions, and can account for identified non-causal confounds. The analysis results in a directed graph whose nodes are brain regions, and whose edges represent information flow. This causal information measure i...
متن کاملProbabilistic Sufficiency and Algorithmic Sufficiency from the point of view of Information Theory
Given the importance of Markov chains in information theory, the definition of conditional probability for these random processes can also be defined in terms of mutual information. In this paper, the relationship between the concept of sufficiency and Markov chains from the perspective of information theory and the relationship between probabilistic sufficiency and algorithmic sufficien...
متن کاملMonotone DAG Faithfulness: A Bad Assumption
In a recent paper, Cheng, Greiner, Kelly, Bell and Liu (Artificial Intelligence 137:4390; 2002) describe an algorithm for learning Bayesian networks that—in a domain consisting of n variables—identifies the optimal solution using O(n) calls to a mutual-information oracle. This seemingly incredible result relies on (1) the standard assumption that the generative distribution is Markov and faithf...
متن کاملThe Information Flow and Capacity of Channels with Noisy Feedback
In this paper, we consider some long-standing problems in communication systems with access to noisy feedback. We introduce a new notion, the residual directed information, to capture the effective information flow (i.e. mutual information between the message and the channel outputs) in the forward channel. In light of this new concept, we investigate discrete memoryless channels (DMC) with noi...
متن کاملInformation-Theoretic Inference of Common Ancestors
A directed acyclic graph (DAG) partially represents the conditional independence structure among observations of a system if the local Markov condition holds, that is if every variable is independent of its non-descendants given its parents. In general, there is a whole class of DAGs that represents a given set of conditional independence relations. We are interested in properties of this class...
متن کامل